Two-transitive Lie groups 

Linus Kramer* 
February 8, 2008 

Abstract 

Using a characterization of parabolics in reductive Lie groups due to Furstenberg, ele- 
mentary properties of buildings, and some algebraic topology, we give a new proof of Tits' 
classification of 2-transitive Lie groups. 

Among many other results, Tits classified in [41] all 2-transitive Lie groups. His proof is based 
on Dynkin's classification of maximal complex subalgebras in complex simple Lie algebras; it 
is long and depends on consideration of various cases. Since the resulting list of groups is also 
long (at least in the afhne case), it is clear that there cannot be a very short proof of the full 
classification. On the other hand, Lie theory has developed since the time [41] was written. 
In particular, Tits himself changed the picture through his theory of buildings (as he pointed 
out, his paper [41] was one of the motivations for him to invent buildings). The language, the 
methods, and the terminology have changed since then, and it is natural to look for a new (and 
shorter) proof of Tits' classification. Note also that the proof presented in [41] IV F 1.2, p. 222, 
does not cover certain real forms of exceptional groups — a footnote on p. 223 asserts that Tits 
found a proof for these cases, too, after the manuscript went into print; see also loc.cit. p. 240. 
The details were never published. 

Almost at the same time as Tits, Borel [4] determined all 1-connected spaces X which 
admit a 2- or 3-transitive Lie group action; however, Borel did not classify the corresponding 
groups. His proof relies on spectral sequences, Freudenthal's theory of ends, and on the results 
of Borel and De Siebenthal about homogeneous spaces of positive Euler characteristic. 

In this paper, we give a complete proof for Tits' classification. The main ingredients are 
a characterization of parabolics in Lie groups due to Furstenberg, elementary properties of 
buildings, some algebraic topology (certainly more elementary than the machinery employed 
in Borel's work [4]), and representation theory of semisimple (compact) Lie groups. 

* Supported by a Heisenberg fellowship by the Deutsche Forschungsgemeinschaft 
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As we remarked before, the only published proof for the classification is [41]. The classifi- 
cation in the affine case is also stated (but not proved) in Volklein [47], and some remarks on 
the strategy of Tits' original proof can be found in Salzmann et al. [36] 96.15 and 96.16. 

Related results for other classes of groups are Knop's classification [26] of 2-transitive 
actions of algebraic groups over algebraically closed fields in arbitrary characteristic (which is 
achieved by quite different methods), and the classification of all finite 2-transitive groups; see 
Dixon-Mortimer [11] 7.7 for a description of these groups. In the course of our classification, 
we recover Knop's result for the special case of complex algebraic groups. 

The main results of the classification £1X6 clS follows. 

Theorem A Let G be a locally compact, a-compact topological transformation group acting 
effectively and 2-transitively on a space X which is not totally disconnected. Then G is a Lie 
group and X = G/G x is a connected manifold. The connected component G° is simple if and 
only if X is compact. 

Note that it is not enough to assume that the group G is locally compact; the full homeomor- 
phism group of any topological manifold, endowed with the discrete topology, satisfies all the 
other conditions of the theorem. 

Theorem B If (G,X) is as in Theorem A, and if X is compact, then X is either the point set 
of a projective space, or the set of all absolute points of a polarity (of index 1) in a projective 
space. In the first case, G is (a finite extension of) the little projective group, and in the second 
case, G is (a finite extension of) the centralizer of the polarity in the little projective group of 
the projective space. 

The projective spaces in question real, complex, quaternionic, or octonionic, and the possibilities 
for the groups G are explicitly determined, see Theorem 3.3. 

Theorem C // (G, X) is as in Theorem A, and if X is noncompact, then X = M. m is a real 
vector space, and G is a semidirect product G = G x k IR m 7 where G x < GL m M is a linear group 
acting transitively on the nonzero vectors. 

We determine explicitly the connected linear groups which act transitively on the nonzero 
vectors in Theorem 6.17. 

The dichotomy that either X is compact and G° is simple, or that X is noncompact and 
G of affine type is proved by ideas similar to Borel's, but with a modest amount of algebraic 
topology. The classification of transitive linear groups depends very much on representation 
theory. 

In case where X is compact, the key ingredient is a characterization of parabolics due to 
Furstenberg. This characterization was used by Burns-Spatzier [9] in order to classify compact 
connected buildings with strongly transitive automorphism groups. Using Furstenberg's result, 
the classification is reduced to a problem about the PU-valued distance in spherical buildings. 
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Here, we need some elementary properties of buildings, (but the proof does not depend on the 
classification of spherical Moufang buildings). We rely of course on the classification of real 
simple Lie groups and their structure theory. 

Outline of the classification. 

The first section collects some basic material on 2-transitive permutations groups. Then 
we show that a 2-transitive locally compact group (which satisfies some additional hypotheses) 
is automatically a Lie group. Up to this point, our proof is more or less the same as Tits' 
original proof; these results can also be found in Salzmann et al. 96.15. After this point, we 
follow a different line than Tits. First we consider the case where the space X which G acts on 
is compact. This case is much easier than the noncompact case, since one can use a convenient 
criterion due to Furstenberg which characterizes parabolics in Lie groups. Using this result, it 
is not difficult to show that G is essentially a simple (noncompact) Lie group, and that X is a 
vertex set of the building A belonging to G. If the real rank of the group G is at least 2, the 
building has to be a projective space, and this leads to a full classification in the compact case. 

The noncompact case is more involved. Here we use some algebraic topology to prove that 
X is contractible if it is noncompact. Once this is proved, it is not difficult to see that X is a real 
vector space, and that G acts through affine-linear transformations. The task is then to classify 
linear Lie groups acting transitively on nonzero vectors in a real vector space. Such a group acts 
transitively on the half-rays in the vector space, and we can use the classification of compact Lie 
groups acting transitively on spheres. In the last section we classify those 2-transitive groups in 
our list which are Moufang sets. This re-proves and generalizes the classification by Kalscheuer, 
Tits, and Grundhofer of all sharply 2-transitive Lie groups. 
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1 Preliminaries 



In this section we collect a few general and simple facts about 2-transitive groups. An action of 
a group G on a nonempty set X is a homomorphism G — > Sym(X) of G into the symmetric 
group of X. For x G X, we denote the stabilizer of x by G x = {g G G | g(x) = x}. For a subset 
(/CGwe put [/ • x = {g(x) | g G [/}. If G acts transitively on X (i.e. if X — G ■ x for some 
x G X) and if < G is a subgroup, then H acts transitively on X if and only if G = G X H. 
The kernel of an action is the subgroup G[x] = f){Gx I x e X}. The action is effective if 
G[x] = 1 (often this is called a faithful action). If the action is effective, G can be identified 
with its image in Sym(X). A transitive action is regular if G x = 1 holds for all x G X. 

The action of G on X is called k-transitive, for |X| > A; > 2, if G acts transitively on the 
set of all Axtuples (x±, . . . ,Xk) G X k with pairwise distinct entries. Clearly, a (k + l)-transitive 
group acts also /c-transitively; in particular, it acts transitively on X. If \X\ > 3, then G acts 
2-transitively on X if and only if the stabilizer G x of x G X acts transitively on X \ {x} for 
every x G X. A transitive action (of G on X) is primitive if the stabilizer G x is a maximal 
subgroup of G, i.e. if G x < H < G implies H = G. 

1.1 Lemma If G acts primitively on X , and if N<G is a normal subgroup, then either N < G x 
(and thus N < G[ X ]), or N acts transitively on X. 

Proof. See eg. Dixon-Mortimer [11] Theorem 1.6A(v). □ 

The following criterion for primitivity is very convenient. Let C I x I be an equivalence 
relation, i.e. xRyRz implies xRz, idx Q R, and i(R) = R, where t(x,y) = (y,x). Let us call 
idx = {(x, x) | x G X} and X x X the trivial equivalence relations. An equivalence relation 
is G-invariant if g(R) = R holds for all g G G (with respect to the diagonal action of G on 
X x X). 

1.2 Lemma A transitive action of G on X is primitive if and only if X x X contains no 
nontrivial G-invariant equivalence relations. 

Proof. See eg. Jacobson [24] Theorem 1.12, or Dixon-Mortimer [11] Corollary 1.5A. □ 

If G is 2-transitive, then G acts transitively on Y = X x X \ idx, hence there are no nontrivial 
G-invariant equivalence relations. Thus, a 2-transitive action is primitive. 

1.3 Lemma Suppose that G acts 2-transitively on X . Then the stabilizer G x is a maximal 
subgroup of G. □ 

1.4 Lemma Suppose that G acts primitively and effectively on X. If T ^ A < G is an abelian 
normal subgroup, then G is a semidirect product G = G x x A. The group A acts regularly on 
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X, and if we identify A with X via the evaluation map ev x : a i — > a(x), then each g G G x acts 
by conjugation, 

ga(x) = gag~ 1 g{x) = gag~ l (x). 

In particular, G x acts transitively on A \ {1} if G acts 2-transitively on X. 

Proof. By Lemma 1.3, A acts transitively on X. A transitive and effective abelian group 
acts regularly. Thus A x — G x D A — 1, and G = G X A is a semidirect product. □ 

In the next lemma, we identify G with its image in Sym(X). The result is used in Section 3. 

1.5 Lemma Suppose that G acts primitively and effectively on X. If G is not cyclic of prime 
order, G ^ Z/p, then the centralizer of G in the symmetric group is trivial, Cens ym (x)(G') = 1. 
In particular, G is centerless, and the normalizer Norg ym (x) (G) is contained in the automorphism 
group of G. 

Proof. Let C = Cens ym (x){G) , and assume that C ^ 1. Then C is a nontrivial normal 
subgroup in the primitive group CG < Sym(X), hence C acts transitively on X by Lemma 
1.3. Let g G G x . Then c(x) = cg(x) = gc(x) holds for all c G C, so g fixes the orbit C ■ x = X 
elementwise. It follows that G acts regularly on X. Since the action is primitive, G has no 
proper subgroups, thus G = Z/p, for some prime p. □ 

2 Two-transitive Lie groups and locally compact groups 

We fix some topological terminology. A homeomorphism is denoted by '=', and a homotopy 
equivalence by Unless stated otherwise, all spaces are assumed to be Hausdorff. An n- 

manifold is a second countable metrizable space which is locally homeomorphic to 1R™. In a 
topological group we assume always that the singletons are closed, so the groups themselves 
are regular topological spaces. A Lie group is a second countable topological group which is 
at the same time a smooth manifold, such that the group operations are smooth maps. A 
transformation group is a pair (G, X) consisting of a topological group G, acting as a group of 
homeomorphisms on a topological space 1^0, such that the map GxX — ► X is continuous. 
If G is a Lie group, we call (G, X) a Lie transformation group. The connected component of 
the identity in a topological group G is denoted G°. 

A space is called cr-compact if it is a countable union of compact subsets. With our 
conventions, Lie groups are a-compact. If (G, X) is a transitive transformation group, and 
if x G X, then the map G — ► G/G x is open, and thus the natural map G/G x — ► X is 
continuous. We need the following sharpening of this fact, cp. Salzmann et al. [36] 96.8. 
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2.1 Proposition Let (G,X) be a transitive transformation group, and let x G X. If G 
and X are locally compact, and if G is a -compact, then the natural map G/G x — > X is a 
homeomorphism; in particular, the evaluation map map ev x : G — ► X, g i — ► g{x) is open. □ 

The Approximation Theorem states that every locally compact group can be approximated by 
Lie groups in the following sense, cp. Salzmann et al. [36] 93.8. Note that every open subgroup 
of G contains the connected component G°, and that G° is a closed normal subgroup of G. 
However, G° need not be open if G is not a Lie group. 

2.2 Approximation Theorem Let G be a locally compact group. 

(1) There exists an open (and a -compact) subgroup V < G such that V/G° is compact. 

(2) Given a subgroup V as in (1), and given any neighborhood U of 1 G G, there exists a 
compact normal subgroup N < V ofV with N C U, such that V/N is a Lie group. □ 

1 



compact 




1 G° V V/G° 1 



V/N 

Lie 



Theorem 2.4 below is essentially due to Tits [41]; the present proof is a simplified version of 
the proof given in Salzmann et al. [36] 96.15. A space is called totally disconnected if the 
component of every point is trivial. The following useful result is due to Eilenberg [13] 3.1. 

2.3 Lemma Let X be a connected space, and let Y = X x X \ idx = {(x, y) \ x, y G X, x ^ 
y}. Let i(x,y) = (y,x). If Y is not connected, then Y has precisely two (necessarily open) 
components A, B, and i(A) = B. □ 

2.4 Theorem Let (G,X) be an effective topological transformation group, where G is o- 
compact and locally compact, and X is locally compact and not totally disconnected. Suppose 
that G acts 2-transitively on X. Then G is a Lie group and X = G/G x is a smooth and 
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connected manifold. Every open subgroup V < G acts primitively on X; in particular, G° acts 
primitively on X. The group G° is noncompact and has at most three orbits in X x X . 

Proof. First we prove that X is connected. Define an equivalence relation R on X by 
setting xRy if x,y are contained in a connected subset. Since X is not totally disconnected, 
R 7^ idx- This relation is G-invariant, since G acts by homeomorphisms on X. Thus R = X xX 
by Lemma 1.2, and hence X is connected. 

Now we show that G° ^ 1. If G° — 1, then G is totally disconnected, and thus zero- 
dimensional, see Hewitt-Ross [21] Thm. 3.5. Therefore, X = G/G x is also zero-dimensional, see 
loc.cit. Thm. 7.11, and X contains arbitrarily small open and closed subsets. This contradicts 
the fact that X is connected. 

Let Y = {(x,y) | x, y G X, x ^ y}. This is a locally compact space, and G acts transitively 
on Y; the evaluation map ev( x ^ y ) : G — ► Y is open by Proposition 2.1. Let V < G be an open 
subgroup. Since V contains the normal subgroup G°, it acts transitively on X by Lemma 1.1. 
For every (x,y) G Y, the orbit evr Xty \(V) = V ■ (x,y) is open. By Lemma 2.3, Y has at most 
two components A, B, so V has at most two orbits A, B in Y. Moreover, t(A) = B, so X x X 
contains no nontrivial ^/-invariant equivalence relation, i.e. the action of V on X is primitive 
by Lemma 1.2. 

Now choose a cocompact open subgroup V < G as in the Approximation Theorem 2.2 (1), 
and choose a small neighborhood U C V of the identity such that U ■ x ^ X. Then U cannot 
contain a proper normal subgroup N <V, since otherwise N ■ x = X . Therefore, V° = G° is 
an open (Lie) subgroup of G by Theorem 2.2, and G/G° is discrete and (by a-compactness of 
G) countable. 

If the open subgroup G° is compact, then its orbits in X x X are compact. Since there are 
at most three different orbits A, B, idx, this would imply that AU B = X x X\idx is compact, 
a contradiction to the fact that X \ {x} is noncompact (because X is connected). □ 

For the remainder of this section, we assume that G is a Lie transformation group acting on a 
connected manifold X = G/G x , and that this action is effective and 2-transitive. 
We need two results about the connectivity of complements of discrete subsets. 

2.5 Lemma Let M be a connected manifold of positive dimension. If M is compact or if 
dim(M) > 2, then M \ {x} is path connected for all x G M. 

Proof. If dim(M) > 2, then #i(M, M \ {x}) = = H (M), and the exact sequence 

— > Hi(M, M \ {x}) — > H (M \ {x}) — > H (M) — > 

shows that H (M \ {x}) = 0, so M \ {x} is also path connected. If dim(M) = 1 and if M is 
compact, then M = S 1 is a circle, and M \ {x} = M. is path connected. □ 



7 



2.6 Lemma If X is compact or if dim (X) > 1, then the connected component G° acts also 
2-transitively on X . 

Proof. The connected component G° < G is a normal subgroup, hence G° acts transitively 
on X by Lemma 1.3. The subgroup G° < G is open, hence {G°) x is open in G x . Let y G X\{x}. 
The evaluation map ev v : g i — ► g(y), G x — > X \ {x} is open by Proposition 2.1; since X \ {a;} 
is connected, ev y ((G°) x ) — X \ {x}, i.e. (G°) x acts transitively on X \ {x}. □ 

2.7 Lemma If G has an ahelian normal subgroup A^l, then A = R™ is a vector group. 

Proof. The group A acts regularly on X by Lemma 1.4, hence A is connected (by Propo- 
sition 2.1). A connected abelian n-dimensional Lie group is of the form R k /Z k x R n ~ fe , for 
some ke{0,...,n}, see Onishchik-Vinberg [33] Ch. 1 §2 Prop. 3, p. 29. If k > 1, then R k /Z k 
contains elements of order / for all / G N. This is not possible by Lemma 1.4 (all elements of A 
different from the identity have to be conjugate under G x ), so k = 0. 

In particular, the existence of a nontrivial proper abelian normal subgroup implies that X is 
noncompact. The converse will be proved in Section 4: if X is noncompact, then G has a 
nontrivial proper abelian normal subgroup. 

2.8 Proposition If G has no nontrivial proper normal ahelian subgroup, then the connected 
component G° is simple. 

Proof. Assume that G has no nontrivial proper abelian normal subgroup. Let VG denote 
the radical of G, i.e. the largest normal connected solvable subgroup, see Onishchik-Vinberg 
[33] Ch. 1 §4 6°. Let 1 < Z < ■ ■ ■ < VG denote the derived series ofVG~. If VG ^ 1, then Z is 
a nontrivial normal abelian subgroup in G, since it is characteristic in \fG. Thus \[G = 1. Note 
also that VG = \/~G°. Therefore G° is semisimple and centerless and thus a direct product of 
simple Lie groups, see Salzmann et al. [36] 94.23. 

If dim(X) > 2 or if X is compact, then G° acts 2-transitively on X by Lemma 2.6. If 
dim(X) = 1 and if X is noncompact, then X = R. By Brouwer's result, no centerless semisimple 
Lie group acts transitively on R, see Salzmann et al. [36] 96.30, so this case cannot occur. 

Assume now that dim(G) > 2, and that G° = G\ x G 2 , with G\ ^ 1 simple. Suppose 
that G2 ^ 1. Then G\ and G2 are normal in G° and thus transitive on X. Suppose that 
gi G G\ fixes x, and that #2 G G2. Then g 2 (x) = g 2 gi(x) = gi(g 2 (^)), so g\ fixes g 2 [x) for all 
g-i G G 2 . Since G 2 ■ x = X, this implies that g\ = 1, i.e. Gi acts regularly on X. We identify 
X with Gi] then the action of Gi on X is the standard left regular action (g,x) 1 — ► gx of 
Gi on itself. The centralizer of this action Cens ym (x)(Gi) is isomorphic to G\ with the action 
(g,x) 1 — > xg^ 1 . It follows that G 2 = G 1 , with the action (gi,g 2 )(x) = gixg^ 1 . The stabilizer 
of 1 G X is the diagonal subgroup {(g, g) \ g G Gi}; its orbits in X are the conjugacy classes of 



8 



G±. But a centerless simple Lie group contains a torus M/Z; in particular, it contains elements 
of arbitrary finite order, and thus infinitely many conjugacy classes. This is a contradiction to 
the 2-transitivity of G°. Therefore Gi — 1 and G = G± is simple. □ 

If we assume in addition that X is compact, then there is a shortcut in the last step of the proof 
above: if G° is a product of at least two simple factors, then each simple factor acts regularly, 
and thus G° is compact, a contradiction to Theorem 2.4. 

3 The case when X is compact 

In this section we classify all pairs (G,X), where G is a Lie group acting effectively and 2- 
transitively on a compact connected manifold X. By 2.6, 2.7, and 2.8, G° is a simple centerless 
Lie group which acts 2-transitively on X. Recall the definition of a parabolic subgroup in an 
arbitrary semisimple Lie group H (of noncompact type). Suppose that 

H — KAU 

is an Iwasawa decomposition for H (the unipotent subgroup U is denoted W in most books 
on Lie groups; we use U, since N has a different meaning with BN-pairs), see Helgason [20] 
Ch. IX §1 Thm. 1.3. Let K = Cenx(^4) denote the if-centralizer of A (this is the reductive 
anisotropic kernel of H, see Tits [42] p. 39). Then 

B = K AU 

is a subgroup, a minimal parabolic subgroup of H, see Helgason [20] Ch. IX §1, Warner [48] 
p. 56. A parabolic subgroup is any subgroup of H which is conjugate to an overgroup of B. 
There are precisely 2 dim ^ conjugacy classes of parabolics in H (including the classes of H and 
B), see Warner [48] Thm. 1.2.1.1. If we put N = Nor H (K A), then (B,N) is a BN-pair of rank 
dim(A) for the group H, see Warner [48] p. 68. 

3.1 Theorem Let H < PSL m+1 IR be a closed semisimple subgroup, and consider the standard 
projective action of PSL m+1 R on the real projective space MP rn . Let X C MP m be a closed 
H-invariant subset. Assume that the following two conditions are satisfied. 

(a) ff 7^ Y C X is closed and H-invariant, then Y = X . 

(b) ff ^ Z C X x X is closed and H-invariant, then Z H idx 7^ 0. 

Then X = H/H x is a homogeneous space of H, and H x is a parabolic subgroup of H. 

Proof. Condition (a) says that the action is minimal in the sense of Furstenberg [14] p. 278- 
279, and condition (b) says that it is proximal; by the very definition, it is what Furstenberg calls 
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projective. By Proposition 4.3. in loc.cit. p. 280, there exists an if-equivariant map from the 
maximal Furstenberg boundary B(H) of H onto X. Since we assumed that H is semisimple, the 
discussion in loc.cit. p. 280 shows that there is an if-equivariant homeomorphism B(H) = H/B 
between the Furstenberg boundary and the coset space H/B, where B < H is a minimal 
parabolic. The space H/ B = Kj K is compact (it is the chamber set of the canonical building 
A(H) associated to H), and thus H x contains a conjugate of B, i.e. H x is a parabolic. □ 

3.2 Proposition If X is compact, and if G is a simple connected Lie group acting effectively 
and 2-transitively on X, then the stabilizer P = G x is a maximal parabolic subgroup of the 
simple group G. 

Proof. Let p denote the Lie algebra of P, with k = dim K (p). The G-normalizer of p 
contains P; by maximality of P, it coincides with P. Consider the adjoint action of G on the 
Grassmann manifold Gr^(g) of all A;- dimensional real subspaces of the Lie algebra g of G. The 
stabilizer of p G Gr^jj) is P, as we just saw. The Grassmannian embeds G-equivariantly into 
the real projective space P(/\ fc g) of the k-th exterior power of the real vector space q. The 
G-orbit of f\ k p e P(/\ k g) is therefore G-equivariantly homeomorphic to X = G/P. Since G 
acts 2-transitively, the conditions (a) and (b) in Furstenberg's Theorem 3.1 are clearly satisfied, 
hence P is a parabolic subgroup, and, by 2-transitivity on X, a maximal one. □ 

Recall that the real rank of a semisimple Lie group H is the real vector space dimension of a, 
where f) = t + a + uisan Iwasawa decomposition of the Lie algebra f) of H. 

3.3 Theorem Let (G,X) be an effective 2-transitive transformation group. Assume that G 
is locally compact and o-compact, and that X is compact and not totally disconnected. Then 
G is a Lie group and the connected component G° is simple and noncompact. There exists a 
subgroup G of the automorphism group of G° such that either G = G°, or G = G. Both G and 
G° act 2-transitively on X. Topologically the space X is either a sphere or a projective space 
over R, C, H, or O. The possibilities for the pair (G°, X) are as follows. 

(a) If the real rank of G° is 1, then X is a sphere, and there are the following possibilities. 



G° 


X 


G 


\G/G°\ 


PEO n ,iM 


gn-1 


PO nil M 


2 


(n 


>2) 


PSU n 'iC 


§2n-l 


pru. M c 


2 


(n 


>2) 


PU nil H 


g4n— 1 


pu n>1 e 


1 


(n 


>2) 


F4(-20) 


§ 15 


F4(-20) 


1 







The simple subgroup PEO ni iR = (PO ni ilR) is the subgroup generated by all Eichler (or Siegel) 
transformations, see e.g. Hahn-O'Meara [19] p. 214. Alternatively, the group PEO ni iR can be 
described as the commutator group of PO n>1 R, or as the connected component (PO^iM) . 
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(b) If the real rank k of G° is at least 2, then X is a projective space of rank k, and there 
are only the following possibilities. 



G° 


X 


G 


\G/G°\ 




RP fc 


PGLfc+iM 


2 


(k 


odd) 




WP k 


PSL/j^x-K 


1 


(k 


even) 


PSL fe+1 C 


CP k 


prL fc+ iC 


2 






PSL fc+1 e 


MP k 


PSL fc+1 e 


1 






E6(-26) 


OP 2 


E6(-26) 


1 


(k 


= v 



Note that there are the following isomorphisms of simple Lie groups: 

PSL 2 M = PSU^R PE0 2il M 
PSL 2 C = PE0 3 ,iR 
PUi,iM = PE0 4i iM 
PSL 2 H = PE0 5 ,iM 

Proof of Theorem 3.3. We know already by 2.6, 2.7, and 2.8 that H = G° is a simple 
centerless Lie group which acts 2-transitively on X, and by Proposition 3.2 the stabilizer P = H x 
is a maximal parabolic subgroup. If the real rank of H is 1, then Table V in Ch. X, p. 518 in 
Helgason [20] shows that (H,X) is a as in (a). Note that a group with a BN-pair of rank 1 is 
the same as a 2-transitive group; thus, each simple Lie group of real rank 1 is a 2-transitive 
group. Of course, this follows also by direct inspection of the list of groups. 

Now suppose that the real rank k of H is at least 2. Then we consider the irreducible 
spherical building A = A(H) of H. Theorem 4.3 in the next section shows that A is the 
building associated to a projective space (of finite rank, as all buildings related to Lie groups 
are of finite rank), and that the parabolic H x is the stabilizer of a point in this projective space. 

Then H is one of the groups PSLfc +1 F, for F = 1R, C, H, and A is a projective geometry 
of rank k over F, or k = 2, and A is the Cayley plane and H = E 6 (_ 26 ). This follows either 
from the topological Fundamental Theorem of Projective Geometry, (as proved by Kolmogorov 
[27], see also Kiihne-Lowen [30] and the survey by Grundhofer-Lowen [17]), or from Cartan's 
classification of simple Lie groups and Tits' classification [42] of their Tits diagrams (unjustly 
sometimes called "Satake diagrams") and their relative diagrams, see Helgason [20] Table VI, 
Ch. X p. 532. In any case, we obtain the list of groups G° and spaces X in (b). 

By Lemma 1.5, G is a subgroup of the automorphism group of H = G°. The outer 
automorphism groups of all simple Lie groups have been determined by Takeuchi [39], see 
also Wolf [49] 8.8 p. 263. In case (b), we have the additional condition that G preserves the 
conjugacy class of P, and thus G has to act by type-preserving automorphisms on the projective 
space. Thus, we obtain the list of groups G given in (a) and (b). □ 
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3.4 Corollary If G is 3-transitive, then X = S™" 1 and G ^ PO n ,iR, for n > 2, or G = 
PEO Ttj iR and n > 3. None of these groups is 4-transitive. 

If the action is sharply 3-transitive (i.e. regular on triples of pairwise distinct points), then 
G PGL 2 R or G = PSL 2 C. 

Proof. If the building A = A(G°) is a projective geometry of rank k > 2, then there 
exists triples of collinear points and triples of noncollinear points, so G cannot be 3-transitive. 
Therefore, a 3-transitive group is one of the groups given in Theorem 3.3 (a). The unitary groups 
PrU n) iC and PU nj iIHI cannot act 3-transitively if n > 2: choose three linearly independent 
vectors u,v,w with (u\v) = (v\w) = (w\u) and (u\u) = (v\v) = (w\w) = (we denote the 
corresponding hermitian form by (x\y) = —x^yo + X\yi + • • • + x n y n ). Then x = u + vi is 
also isotropic, but no semilinear map fixing the subspace spanned by {u,v} can move the 
subspace spanned by x to the subspace spanned by w. There is a more geometric way to see 
this, which is also valid for the group F 4 (_ 20 ). These groups can be viewed as subgroups of 
PrL n+ iC, PGL n+ iH, acting on the set Q of absolute points of a hyperbolic polarity in the 
projective geometry (for the group F 4 / 2 o), the corresponding projective space is the Cayley 
plane PG 2 (0), see [36] Ch. 1). If we fix two distinct points x,y G Q, then we fix the line L 
joining these two points. Now L n O is a proper subset of Q, and L n Q ^ {x, y} if n > 2; in 
fact, L n Q = § dim aP-i for F = R, C, M, O. This excludes the unitary groups and F 4( _ 20 ). 

The group PE0 2j iIR = PSL 2 IR is not 3-transitive; the remaining orthogonal groups are 
3-transitive, as can be checked. Finally, none of these groups is 4-transitive: Let p,q,r,s& Q 
be four distinct points such that the lines pq and rs intersect in an interior point of Q. Then 
r, s cannot be moved to two points r', s' (fixing p, q) such that r's' intersects pq in an exterior 
point. If the action is sharply 3-transitive, then dim(G) = 3dim(X), and thus only the groups 
PGL 2 R and PSL 2 C remain. □ 

3.5 Corollary If X is a complex homogeneous space (i.e. if X is a complex manifold and if 
G preserves the complex structure), then X = CP™ and G = PSL n+ iC. 

Proof. Suppose that X is a complex manifold, and that the G-action preserves the complex 
structure. Then dim(X) is even. In case (a) of Theorem 3.3, X = E> 2k is a sphere, and 
G° = PE0 2 fc + i i iIR. The compact subgroup K = SO(2k + 1) acts transitively on X, and the 
isotropy representation of K x on the tangent space T X X = R 2fc is the standard action of SO(2/c). 
If k > 2, then this action is not complex linear. Thus k — 1 and G° = PEO^iR = PSL 2 C. 

In case (b) of 3.3, X is an even-dimensional real projective space or a complex or quater- 
nionic projective space. Again we consider the isotropy representation of K x for a suitable 
compact subgroup K < G°. For RP 2fc , we have K = SO(2/c + 1), and K x = 0(2k) acts in the 
standard way on T X X ^ R 2fc . This action preserves no complex structure on the tangent space, 
not even for k — 1. For X = HP n we take K = Sp(n + 1); then K x = Sp(n) • Sp(l) acts in 
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the standard way (from the left and right) on T X X = H n ; again, there is no invariant complex 
structure for all k > 1. Thus the only possibility is that G° = PSL n+1 C in the standard action 
on CP™. Complex conjugation is not C-linear, hence G = G°. □ 

4 Digression: two-transitive actions on buildings 

In this section we consider the following situation. A is a building (thick, and of arbitrary, 
possibly infinite rank), and G is a group of (type-preserving) automorphisms of A. We assume 
that the action of G on one type of residues of A is 2-transitive. This is a purely combinatorial 
problem, and we make no topological assumptions. After I had finished a first version of this 
section, Bernhard Miihlherr pointed out to me that the book by Brouwer-Cohen-Neumaier [8] 
contains related results. 

Recall that a Coxeter diagram is an undirected labeled graph whose nodes are labeled by 
some set I. For each pair i,j G I, there is a number = rriji G N U {oo}, with ma = 1, and 
rriij > 2 if % ^ j. If the nodes i,j are not adjacent (joined by an edge), then = 2. If the 
nodes i,j are adjacent, and if > 4, then the edge joining them is labeled with the number 
rriij. The corresponding Coxeter group is the group generated by a set S = {si \ i G 1} of 
generators labeled by I, with relations (siSj) mij = 1 (for ^ oo). The pair (W, S) is called a 
Coxeter system. The Coxeter system is irreducible if its Coxeter graph is connected. Reducible 
Coxeter systems are products (in a natural sense). For J C 7, the subgroup generated by 
Sj = {sj | j G J} is denoted Wj; one can prove that the pair (W, Sj) is again a Coxeter 
system. We refer to the books by Bourbaki [5] and Humphreys [22], and to Ch. 2 in Ronan 
[35] . Recall the Coxeter diagram A k which is defined as 

• • • • • • • • • 

12 3 k-2 k-1 k 

We denote the limit of these Coxeter diagrams (as k goes to infinity) by A w ; the corresponding 
Coxeter diagram is thus 

• • • • • • 

1 2 3 4 5 6 

The next result can also be extracted from Cooperstein [10], cp. Brouwer-Cohen-Neumaier [8] 
Thm. 10.2.3 p. 300. 

4.1 Proposition Let (W, {sj | i G /}) he an irreducible Coxeter system, where I has (possibly 
infinite) cardinality k. Let J be a subset of I and assume that W acts 2-transitively on W/Wj. 
Then k < K , and either the Coxeter system is of type with J = {1} or J = {k}, or of 
type A u , and J — I \ {1} (the nodes of the diagram are labeled as above). 
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Proof. The proof is very simple. We divide it into four basic steps. 

(1) J has corank 1, i.e. I \ J is a singleton. 

Since Wj has to be a maximal subgroup, / \ J is a singleton, which we denote by {1}. 
Thus we have W = Wj U WjSiWj. Now we use the standard description of shortest double 
coset representatives: every element w which has the property that all reduced expressions of 
this element start and end with s 1 represents a unique double coset WjwWj. 

(2) In the Coxeter diagram, the node 1 has at most one neighbor. 

Assume that node 1 has two different neighboring nodes, say 2, 3. Then S1S2S3S1 is a 
reduced word representing a third double coset, Wjs^ssSiWj (note that the order of S2S3 is 
not important for this argument). 

(3) No node in the Coxeter digram has more than two neighbors. 

Assume otherwise; let i 7^ 1 be a node with three different neighbors. Let 1-2-3— • --{i — l)-i 
be a shortest path in the Coxeter diagram, and let j, k be distinct neighbors of node i different 
from i — 1. Then sis 2 • • • s^iSiSjSkSiSi-i ■ ■ ■ Si is a reduced word which represents another 
shortest double coset representative. This implies already that / is finite or countable, and 
that the Coxeter diagram of W is a string diagram (with at least one end) . 

(4) The Coxeter diagram has no multiple bonds, rriij < 3. 

Assume that 1-2— • • -i-j is a shortest path in the diagram, and that i, j are joined by an edge 
labeled m^- > 4. Then SiSj has order at least 4. The reduced word S1S2 • • • Si-iSiSjSiSi-i • • • si 
represents another shortest double coset representative. 

By (1) - (4), the diagram has only simple bonds and does not branch, and 1 is an end 
node. □ 

Now we consider 2-transitive actions on buildings. Since we allow infinite rank, we view build- 
ings as chamber systems, see Tits [45] or Ronan [35]. A chamber system is a set C, endowed 
with a collection {~j | i G /} of equivalence relations. For a subset J C J, the equivalence 
relation generated by {~j | j G J} is denoted ~j. Given a Coxeter system (W, {sj | i G /}), a 
building 

A = (C,{~i\ieI},6,W,{ Si \ieI}) 

is a chamber system C, endowed with a metric 5 : C x C — > W taking its values in a Coxeter 
group (subject to certain axioms which can be found eg. in Ronan's book [35] Ch. 3). An 
automorphism of a building (in this sense) is a permutation of C which preserves the equivalence 
classes (and thus the metric 5). A J -residue in a building is a J-equivalence class. 

For i G /, let Vi denote the set of all residues of type / \ {i} in the building. Define an 
incidence relation * as follows: two residues are incident if their intersection is nonempty. The 
datum T = ({Vi}i e i,*) determines a diagram geometry. Suppose now that the building is of 
type A K , for 2 < k < uo. Put V = V\ and C = V 2 (the nodes are labeled as before). 
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4.2 Theorem If k > 2, then the geometry (V, C, *) obtained from an A K -building is a projec- 
tive space. 

Proof. The proof indicated by Tits in [45] p. 540 applies (also for the case k = oo). □ 

4.3 Theorem Let A be a thick irreducible building over some type set I, let G be a group of 
type preserving automorphisms of A acting 2-transitively on the collection of all J-residues, for 
some J. Then A is of type or A w ; the corresponding diagram geometry can be identified 
with a projective space (possibly of inhnite rank), and G acts 2-transitively on the points of 
this space. 

Proof. Let X, Y, Z be distinct J-residues. Since the stabilizer of X is a maximal subgroup, 
J has corank 1, i.e. / \ J is a singleton. There is a unique element w G 5(X x Y) C W 
such that £(w) minimizes the lengths £(5(x, y)), where x, y run through all chambers in X, Y , 
see Scharlau-Dress [12]. Since the action is 2-transitive on the J-residues, the same w gives 
the minimal distance for elements in Z. Thus W = Wj U WjwWj; in particular, W acts 
2-transitively on W/Wj. By Proposition 4.1, A is of type A k or A w . □ 

5 The case when X is noncompact 

In this section we assume that G is a 2-transitive effective Lie transformation group acting on 
X, and that X is noncompact. We show that in this case X = R n is a real vector space, and 
that G splits as a semidirect product G = G x x lR n . In view of Lemma 2.7 and Proposition 2.8, 
it suffices to prove that G° cannot be simple if X is noncompact. We need some more facts 
about transformation groups. For a group H acting on a set X, we denote the fixed point set 
by 

X H = {x E X | H-x = {x}}. 
Recall the Malcev-Iwasawa Theorem, cp. Salzmann et al. 93.10. 

5.1 Malcev-Iwasawa Theorem Let G be a connected locally compact group. 

(1) There exists a maximal compact subgroup K < G. If L < G is a compact subgroup, 
then gLg~ x < K for a suitable g G G. 

(2) There exist closed subgroups A 1 ,...,A m < G isomorphic to (R, +), such that G is 
directly spanned as G = AiA 2 ■ ■ ■ A m K (i.e. every g G G can in a unique way be written as a 
product g — a±- • • a m k, with ai G A-i and k G K). In particular, G is homeomorphic to K x R m , 
and G/K = W 71 is contractible. 

We also need the following result. 



15 



5.2 Lemma Let K be a compact connected Lie group. Then K is abelian if and only if 

7T 3 (K) =0. 

Proof. It is a well-known result that ir 3 (H) = Z holds for every almost simple compact 
Lie group H, see Onishchik [32] §17 Theorem 2, p 257. A compact connected Lie group is 
topologically a product of compact almost simple Lie groups and a torus; it is abelian if and 
only if no nontrivial simple factors exist. □ 

5.3 Corollary Let G be an almost simple Lie group. If G is not locally isomorphic to SL 2 M, 
then ir 3 (G) ^ 0. 

Proof. Since we consider only the third homotopy group, we may assume that G is center- 
less. Let K < G be a maximal compact subgroup. A direct inspection of Table V in Helgason 
[20] Ch. X, p. 518 or Onishchik- Vinberg [33] Table 9 p. 312 shows that K is not abelian, pro- 
vided that G 7^ PSL 2 IL Since we have a homotopy equivalence K ~ G by the Malcev-Iwasawa 
Theorem 5.1, the claim follows. □ 

For a proof of Whitehead's Theorem below see Spanier [38] Ch. 7 Sec. 6 Corollary 24. 

5.4 Theorem Let f : X — > Y be a continuous map. If X and Y have the homotopy types 
of CW-complexes (eg. if X and Y are manifolds, see Bredon [6] V 1.6) and if the induced map 
tt(/)» : 7r »(^) — > n »(y) JS an isomorphism, then f is a homotopy equivalence. 

Combining these two results, we obtain the next lemma. 

5.5 Lemma Let Z = H/H z be a homogeneous space of a Lie group H. Assume that H and 
H z are connected. Let K < H be a maximal compact subgroup, such that K z is a maximal 
compact subgroup of H z . Then there is a homotopy equivalence K/K z ~ H/H z . 

Proof. The spaces H/K and H z /K z are contractible by the Malcev-Iwasawa Theorem 5.1. 
A fibration of CW-complexes with contractible fibres (with a contractible base, resp.) induces 
a homotopy equivalence between the base and the total space (the fibre and the total space, 
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resp.) by Whitehead's Theorem 5.4. Consider the fibrations 

K/K z 




H z /K z H/K z — H/H z 



H/K 

The maps / and g are homotopy equivalences, and so is their composite gf. □ 

The following theorem can be proved under much weaker assumptions; however, the present 
version, which is taken from Bredon's book [6] suffices for our purposes. 

5.6 Theorem Let X be a connected noncompact n-manifold, and let K be a compact Lie 
group acting smoothly or locally smoothly on X. Assume that K has an (n — 1) -dimensional 
orbit. Then there are only the following two possibilities. 

(a ) X/ K = R and every K -orbit is principal. In this case, X decomposes as X = Kj K x x R, 
with trivial K-action on M. 

(b) X/K = [0, oo), and there exists precisely one nonprincipal orbit K ■ z = Z = K/K z . 
Let K x < K z be the stabilizer of a point x e X \ Z. Then K z /K x = S m is a sphere, and X is 
equivariantly homeomorphic to the (m + l)-vector bundle bundle associated to the orthogonal 
sphere bundle K z /K x — ► K/K x — ► K/K z . 

Proof. For the concept of a locally smooth action see Bredon's book [6] Ch. IV. A smooth 
action of a compact Lie group is locally smooth by loc.cit. VI 2.4; by IV 3.1, principal orbits 
exist. Our claim is thus a slight reformulation of loc.cit. IV 8.1. □ 

5.7 Corollary If n > 2 and if K has a fixed point, then X is equivariantly homeomorphic to 
euclidean space R n , and K acts transitively on S n_1 . 

Proof. In this case, Z = {z}. A vector bundle over Z is the same as a real vector space 
]R n . □ 

The proof of the next lemma was kindly pointed out by R. Lowen. 
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5.8 Lemma Let M be a connected manifold of dimension n = dim(M) > 3. If D C M is 
a closed discrete subset, then the inclusion M\D c induces an isomorphism of funda- 
mental groups. 

Proof. Assume first that D = {xo} is a singleton. Let <fi : (IR n ,0) — > (M,xo) be a 
coordinate chart and put B £ = {(f)(x) | \x\ < e}. Then M = B 2 U (M \ B\), and M \ B\ ~ 
M \ {x }. Moreover, B 2 fl (M \ Bi) — B 2 \ B x ~ S n_1 is 1-connected, and we can apply the 
Seifert-Van Kampen Theorem to the diagram 



B 2 \B 1 1 




M 7Ti(M) 



to conclude that 7ri(M\{x }) — > ^i(M) is an isomorphism (see for example Bredon [7] Ch. Ill 
Cor. 9.5). An easy induction shows that 7ri(M \ D) — > ^i(M) is an isomorphism, provided 
that D is finite. 

Assume now that D C M is an arbitrary closed discrete set not containing the base 
point. We show first that 7Ti(M \ D) — > tti(M) is surjective. So let j3 : [0, 1] — > M be a path 
representing an element [/^^(m) of n 1 (M), and let U be a relatively compact open neighborhood 
of the image of (3. Then U fl D is finite, and by our previous discussion, 7Ti(C/ \ -D) — >■ ^\{U) 
is an isomorphism. Let a : [0, 1] — > U \ D be a path representing an element [ckJ^^d) in 
7ii(U\D) whose image [a]^^) in iri(U) represents the same element [/SJ^^) as f3. Then a and 
(3 represent the same element in 7Ti(M), i.e. [o^m) = [^^(m)- 

Next we prove that the map is injective. Let 7 be a path representing an element of 
tii(M \ D), and assume that its image [tI^m) is homotopic to the constant path. Fix such a 
homotopy F : [0, 1] x [0, 1] — > M, and let V be a relatively compact open neighborhood of 
the image of F. Then 7 represents the identity element in 717 (V). By our previous discussion, 
[7] 7ri (y\D) is also the identity element, and thus [7] 7ri (M\_D) is the identity element. □ 

5.9 Definition Let G be a connected Lie transformation group acting transitively on a man- 
ifold X. Assume that the following holds: for every x G X, the fixed point set X Gx = 
{y£X\G x -y = {y}} of G x discrete (and necessarily closed), and G x acts transitively on its 
complement X \ X Gx . Then we call G almost 2-transitive on X. 
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5.10 Proposition Let G be a connected Lie group acting almost 2-transitively on a noncom- 
pact 1-connected n-manifold X, with n > 3. Then G acts 2-transitively on X , and X = IR n is 
contractible. 

Proof. By assumption, ir (G) = 1 = n 1 (X). From the exact sequence 

^X) -7T (Gy ^TTo(G) 

we see that G x is connected, and X \ X Gx is 1-connected by Lemma 5.8. Let y E X \ X Gx , so 
G x /G x ^ y = X \ X Gx . The exact sequence 

tti(X \ X G *) n (G Xj y) ir (G x ) 

shows that G XjV is connected. We choose a maximal compact subgroup K < G such that 
K x < G x and K XiV < G x>y are maximal compact subgroups. Note that each of these groups is 
connected. 

Let F 2 denote the field with two elements. Then H k (X; F 2 ) = = H k (X \ X Gx ;¥ 2 ) for all 
k > n, because X and X\X Gx are noncompact manifolds, see Bredon [7] Ch. VI Cor. 7.12 e.g. 
Moreover, dxm(H n (X, X \ X Gx ; F 2 )) = \X Gx \ (by excision), and the exact sequence 

H n (X; F 2 ) H n (X, X \ X Gx ;¥ 2 ) H n ^(X \ X Gx ;¥ 2 ) 

shows that dim(H n _i(X \ X Gx ;¥ 2 )) > \X Gx \. But X \ X Gx ~ K x /K x>y is homotopy equiva- 
lent to a compact connected manifold of dimension strictly less than n by Lemma 5.5. Thus 
dim(K x / K XjV ) = n — 1, and H n ^i(K x / K x , y \¥ 2 ) = ¥ 2 . In particular, \X Gx \ = 1, and thus G x 
acts transitively on X \ {x}, i.e. G is 2-transitive. The orbit K x -y = K x /K x ^ y has codimension 
1 in X. By Corollary 5.7, X = R n . □ 

5.11 Proposition Let G be a 2-transitive Lie group acting on a noncompact connected man- 
ifold X of dimension dim(X) = n > 3. Then X is 1-connected, and thus by Proposition 5.10 
X = R n is contractible. 

Proof. Replacing G by the universal cover of its connected component, we may assume 
that G is 1-connected. Thus we have an isomorphism ni(X) = n (G x ). Let H = (G x )°. The 
exact sequence 

7ri(G) MG/H) no(H) 

shows that Z = G/H is 1-connected, and Z = G/H — v —^ G/G x = X is the universal covering 
of X. Let F = p~ x (x). We claim that H acts transitively on Z \ F. Let z G Z \ F. Then 
p(H ■ z) — H ■ p(z) — X \ {x}, because H is open in G x and because X \ {x} is connected. 
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Thus the if -orbits in Z \ F are n- dimensional and hence open. But Z \ F is connected, so 
H • z — Z\F. Thus G acts almost 2-transitively on G/H. By Proposition 5.10, the action is 
in fact 2-transitive and thus H = G x . □ 



Finally, we consider the low- dimensional cases. 

5.12 Lemma Let G be a 2-transitive Lie group acting on a noncompact connected manifold 
X of dimension dim(X) = n < 2. Then X = W l is contractible. 

Proof. The only noncompact connected 1-manifold is IL 

If dim(X) = 2, then H = G° acts 2-transitively by Lemma 2.6. Assume that X is a 
noncompact surface. If H is not simple, then X = M? by Lemma 2.7. Assume that H is 
simple. Since H acts 2-transitively, dim(iJ) > 4, and in particular H ^ PSL2IR. Let K be a 
maximal compact subgroup of H. Since H 7^ PSL 2 M, the group K is not abelian, and thus 
tt 3 (H) = ir 3 (K) 7^ by Corollary 5.3. Since X is a noncompact surface, the only homotopy 
group of X which is possibly nontrivial is the fundamental group. The exact sequence 

7T 4 (X) 1T 3 (H X ) 7T 3 (H) 7T 3 (X) 

shows that ir 3 (H x ) 7^ 0, hence H x contains a connected nonabelian compact subgroup L < (H x )° 
by Lemma 5.2. Choose y £ X \ X L . Then L ■ y = S 1 is a circle, and X = M 2 by Corollary 5.7. 
(We will see below that this case is impossible.) □ 

5.13 Corollary IfG is an effective 2-transitive Lie group acting on X and if X is noncompact, 
then G° is not simple. 

Proof. By Proposition 5.11 and Lemma 5.12, the space X is contractible. Assume that 

H = G° is simple. Since X is contractible, the stabilizer H x is connected, and H x c *-H is 

a homotopy equivalence by Whitehead's Theorem 5.4. Thus H x contains a maximal compact 
subgroup K of G. But the maximal compact subgroups in a noncompact simple Lie group are 
maximal subgroups, see Helgason [20] Ch. VI Ex. A3(iv) p. 276 and p. 567, hence H x = K, so 
X = H/K is a Riemannian symmetric space, and H preserves the metric; in particular, H has 
infinitely many orbits on X x X, contradicting Theorem 2.4. □ 

Combining this result with our previous analysis, we obtain the following final result for the 
noncompact case. 

5.14 Theorem Let G be a locally compact and a -compact group acting effectively and 2- 
transitively on a locally compact, noncompact, not totally disconnected space X. Then G is a 
Lie group and has a normal vector subgroup M. n < G acting regularly on X = W 1 . Moreover, 
G = G x x R™ is a semidirect product ofW 1 and a point stabilizer G x . 
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Ifn=l, then G = AGUR = GL ± R x R. 

If n > 2, then the connected stabilizer (G x )° is a reductive linear Lie group acting transi- 
tively on the nonzero vectors ofR n . All possibilities for (G x )° (and thus for G° = {G x )° ix R n ) 
are determined in Theorem 6.17. None of these actions is 3-transitive. If G acts sharply 2- 
transitive (i.e. if the two-point stabilizers are trivial), then n = 1,2,4, and G x = (G x )° is one 
of the groups given in Corollary 7.4. 

If X is a complex manifold and if G preserves the complex structure, then G° is one of the 
groups given in Theorem 6.17 (d). □ 

In particular, we obtain Knop's result [26] for characteristic 0: 

5.15 Corollary If G is a complex connected Lie group which acts complex analytically and 
2-transitively on a complex manifold X , then either 

(a) X^ CP n and G = PSL n+1 C, or 

(b) X = C n and G x = SL n C or G x = GL n C, or 

(c) X = C 2n and G x Sp 2n C or G x Sp 2n C • C*. 
In case (b) or (c), G = G x k C n . 

Note however that in the case X = C ra , there exist several real, noncomplex 2-transitive groups 
by Theorem 6.17. 

6 Transitive groups acting on M m \ 

In this section we classify all closed linear Lie groups acting transitively on the nonzero vectors 
of a finite dimensional real vector space. The main ingredients in our proof are the classification 
of compact connected linear Lie groups acting transitively on spheres, and the representation 
theory of compact Lie groups. 

6.1 Theorem Let m > 2, and let K < SO(m) be a closed connected subgroup. If K acts 
transitively on the sphere § m_1 = {x G R m \ \x\ = 1}, then K is (up to automorphisms of 
SO(m) ) one of the following groups. 
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IS. 


IK 


m 






i i 


SU(n) 


c n 


2n 


U(n) 


c n 


2n 


Sp(n) 


M n 


An 


Sp(ra) -U(l) 


W 1 


An 


Sp(n) • Sp(l) 


U n 


An 


G 2 


Pu(O) 


7 


Spin(7) 


o 


8 


Spin(9) 


o©o 


16 



Besides the standard inclusions between the classical groups (eg. Sp(fc) < SU(2A;) < U(2k) < 
SO(Ak) etc.), the inclusions in the special dimensions 7, 8, and 16 are as follows (note that 
SU(4) = Spin(6) and Sp(2) = Spin(5)J. 

7 8 16 



SO(7) 



G 9 



SO(8) 



Spin(7) 



SU(4) 



Sp(2) 



SO(16) 




Spin(9) 



Sp(2) 




SU(8) 



Sp(4) 



All exceptional phenomena in dimensions 7, 8, 16 are related to the Cayley plane PG2O; see 
Salzmann et al. [36] Ch. I. 

Proof. We could use the classification of compact Lie group acting transitively on spheres, 
due to Montgomery-Samelson [31] and Borel [2, 3]; see also Poncet [34], Besse [1] 7.13, Onishchik 
[32] §18 Theorem 3 (i), Salzmann et al. [36] 96.20-23, and Kramer [28] Ch. 6. However, we 
actually need only the classification of transitive subgroups of SO(n), which is much easier and 
follows directly from Onishchik's classification of factorizations of SO(n), see [32] p. 227 and 
[15] II §4.5, p. 144. □ 



6.2 Corollary Suppose that m > 3. Then the commutator group K' 
transitively on S m_1 . 



[K, K] acts also 



22 



Proof. This follows either by direct inspection of the list, or by a simple homotopy-theoretic 
argument, see Onishchik [32] §5 Prop. 9 or Grundhofer-Knarr-Kramer [18] Lem. 1.3. □ 

6.3 Proposition Let V be a finite dimensional real vector space of dimension m > 3, and let 
H < GL(V) be a closed connected group. Suppose that H acts transitively on the nonzero 
vectors of V. Then H is reductive, and consequently H' = [H, H] is a semisimple subgroup of 
SL(V). If K < H is a maximal compact subgroup and if \ — \ is a K-invariant norm on V, then 
K and [K, K] act transitively on the sphere § m_1 = {a? G V" | |rc| = 1}. Consequently, K is one 
of the groups given in Theorem 6. 1 . 

Proof. Clearly, H acts irreducibly on IR m , and thus H is reductive, see Salzmann et al. [36] 
95.2, 95.6. Let K < H be a maximal compact subgroup, let | — | be a K-invariant norm, and 
let v be a vector with \v\ = 1. There is a homotopy equivalence K/K v ~ H/H v = W 11 \ {0} 
by Lemma 5.5. Thus K/K v ~ S m_1 , and in particular dim.{K / K v ) — m — 1. By domain 
invariance, K ■ x = S m_1 , and by Corollary 6.2, the commutator group [K, K] is also transitive 
on S™- 1 . □ 

6.4 Corollary There exists a simple subgroup H\ < H such that K n H\ acts transitively on 

Proof. By Proposition 6.3, [K, K] < [H, H] acts transitively on S m . Theorem 6.1 shows in 
each case that K has a simple factor which acts transitively on S m_1 . We can choose a simple 
factor Hi < [H, H] containing this simple factor of K. □ 

Now we determine all possibilities for the semisimple group [H, H]. The following facts will be 
useful. 

6.5 Theorem Let H < SL m M be a closed semisimple subgroup. After conjugation with some 
element in SL m IR, the Cartan decomposition of5l m M = so(m)(Bp is also a Cartan decomposition 
oft), 

^ = (fjnso(m)) © (fjnp). 

Proof. See Onishchik- Vinberg [33] Ch. 5 Theorem 4, p. 261. □ 

6.6 Lemma Let q = t © p be a Cartan decomposition of a simple noncompact Lie algebra. 
Then 

[p,p]=t 

In particular, if 1 < g is a subalgebra containing p, then I = Q. 

Proof. Let f) = [p,p] < t. Using the Jacobi identity, it is clear that [t, h] < h. Since 
[p, f)] < P, we conclude that f) + p is an ideal in $j, hence f)+p = $j = {! + p. This is a direct sum 
decomposition with f) < t, therefore f) = t. □ 
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We use the following facts from representation theory. We rely on Tits [43] and [44], on the 
Reference Chapter in Onishchik-Vinberg [33], and on Chapter 9 in Salzmann et al. [36]. 

6.7 Remarks on representation theory Associated to a real semisimple Lie group G is its 
weight lattice; every weight ix is a linear combination with integral coefficients of the so-called 
fundamental weights 7Tj. The dominant weights (i.e. the weights where all coefficients are 
nonnegative) correspond bijectively to the equivalence classes of (finite dimensional) complex 
irreducible representations of G. We denote the complex irreducible module associated to 
the dominant weight ix by R(n). The Galois group of C/R acts on the weight lattice via 
7r i — > 7f; if it = fr then R(ir) admits a real or a quaternionic structure (i.e. a complex 
semilinear endomorphism with 2 = id resp. <f) 2 = —id). There is a corresponding map 
/3 from the invariant weights into the Brauer group Br(R) = {1R, H}. There are three types of 
real irreducible representations of G: 

(1) If 7r ^ 7f , then 7r is of complex type and m.R(tt) = R(ir) is a real irreducible G-module. 

(2) If 7r = 7f and (3 W = R, then tx is of real type. There exists a unique real irreducible 
G-module rR(tt), and R(ir) = rR(it) <g> C. 

(3) If 7r = 7f and = H, then 7r is of quaternionic type. In this case, rR(tt) = R(ff) is a 
real irreducible module, and R(tt) admits a quaternionic structure. 

The Reference Chapter in Onishchik-Vinberg [33] contains tables which we will frequently use 
to explain how exterior powers f\ k V, symmetric powers S k V, and tensor products V® k of 
complex irreducible modules V can be decomposed. 

We fix some notation. We denote the ring of n x n-matrices with entries in F by F(n). 
The notation for Lie groups and algebras is as in Onishchik-Vinberg [33]. In particular, we 
denote the quaternion unitary group U n H by Sp(n). Let 

S{n) = {X e R(n) | X T = X} and S (n) = {X e S(n) \ tr(X) = 0} 

The Cartan decomposition of s[ m M is s[ m IR = $o(m) © S (m). 

6.8 Lemma Let H < SO(n) be a subgroup. Then there is a standard action of H on S (n) 
which is given by X i — >■ hXh T . The following actions of subgroups of SO (n) on S (n) are 
M>-irreducible for n > 2: 



H 


n 


SO(n) 


n 


G 2 


7 


Spin(7) 


8 
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Proof. For the orthogonal groups, this follows from the fact that every quadratic form can 
be diagonalized by conjugation with an element of SO(n). If 7^ U < So(n) is an SO(n)- 
invariant subspace, then U intersects the space A consisting of diagonal traceless matrices 
nontrivially. Moreover, the SO(n)-stabilizer of A induces the symmetric group Sym(n) on the 
diagonal matrices. Thus U contains A, whence U = S (n). (The action of SO(ra) is in fact a 
polar action; it is the isotropy representation of the Riemannian symmetric space SL n IR/SO(n). 
The principal orbits are isoparametric submanifolds; the orbit types form the building A(SL n R), 
i.e. the real projective geometry of rank n — 1). 

For the groups G2 and Spin(7) we use representation theory. The complex module corre- 
sponding to the action of G 2 on the set Pu(O) = M 7 of pure octonions is R(ni), and 

5(7) <g> C S 2 R{7n) = R(2tt 1 ) © C. 

The Galois group of C/R act trivially on the weights, and all representations of G2 are of real 
type, see Tits [43] p. 42. Thus 5*0(7) = rR(2tti) is a real irreducible G 2 -module. 

The reasoning for Spin(7) is similar. The complex module for the 8-dimensional represen- 
tation is R(ir 3 ), and S 2 R(tc 3 ) = R(2n 3 ) © C, see Onishchik-Vinberg [33] p. 301. This module is 
of real type, see Tits [43] p. 31, hence rR(2tt 3 ) © C = R(2n 3 ). Thus S (S) = rR(2tc 3 ) is a real 
irreducible Spin(7)-module. □ 

Now we determine all closed noncompact semisimple subgroups of SL m R which contain one of 
the compact groups in Theorem 6.1. 

6.9 Proposition Let H < SL m M be a closed noncompact semisimple subgroup, for m > 3. If 
SO(to) < H, then H = SL m R. If m = 7 and if G 2 < H, then H = SL 7 R. If m = 8 and if 
Spin(7) < H, then H = SL 8 M. 

Proof. Let f) denote the Lie algebra of H. We may assume that f) < sl m M = so(n) © So(n) 
is embedded as in Theorem 6.5. Let K = SO(m) n H. Then f) D So(m) is a nonzero ii'-module. 
Since we assume that K = SO(m) (resp. that G 2 < K or that Spin(7) < K), it follows from 
Lemma 6.8 that f) fl S (m) = S (m), and thus f) = sl m R by Lemma 6.6. □ 

The case of Spin(9) acting on 0©0 is different. Consider the affine Cayley plane AG 2 0, and let 
H < GL 16 IR denote the stabilizer of the origin in its collineation group. Then H = (Spin 91 lR) , 
and H fl F 4 = Spin(9), see Salzmann et al. [36] Sec. 15, in particular 15.6, and p. 628. 

6.10 Proposition Let H < SL 16 1R be a closed noncompact semisimple subgroup with Spin(9) < 
H. Then H is one of the groups (Spin 91 M)° or SL 16 M. 

Proof. The idea is to decompose the Spin(9)-module 5 , (16) into irreducible submodules. 
We denote the Lie algebra of Spin(9) < SO(16) by spin(9) < so(16). Let V = O © O denote 
the 'natural' Spin(9)-module. Then 5(16) S 2 V. 
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The Galois group of C/R acts trivially on all weights of Spin(9), and all fundamental 
representations of Spin(9) are of real type, see Tits [43] p. 31. We label the fundamental weights 
of Spin(9) in the standard way (as in Onishchik-Vinberg [33] p. 293 or in Tits [43] p. 30). The 
complex module -R(7r 4 ) associated to the weight 7r 4 is the Spin-module, R(tt^) = V © C. By 
Onishchik-Vinberg [33] p. 301, we have a decomposition 

S 2 R(-k a ) R(2tt 4 ) © #(tti) © C 

Therefore we obtain a decomposition into real irreducible modules 

S 2 V = Rj R(27t 4 ) © Kj R(tt 1 ) © R and 5 (16) = Rj R(27r 4 ) © Rj R(tt 1 ) 

The dimensions of the modules m.R(tti) = R 9 , and K _R(27r 4 ) are 9 and 126, respectively, see 
Onishchik-Vinberg [33] p. 301. 

Now suppose that H < SLi 6 R is embedded as in Theorem 6.5. Since Spin(9) < H, the 
Lie algebra f) of H is a Spin(9)-module. Consider the nonzero Spin(9)-module M — f) n5o(16). 
If M = Sq(16), then f) = sIiqR by Lemma 6.6. Since we know that SO^iR < sl^R, the 
case M = rR(7Ti) is possible, and [M, M] = spin(9) in this case, so f) = sOg^M is uniquely 
determined. Finally, suppose that M = ffi i?(27r 4 ). Then dim(spin(9) + M) = 162; no such 
semisimple Lie algebra (with spin(9) as maximal compact subalgebra) exists. □ 

Now we consider semisimple groups which contain SU(n). Let H(n) = {X e C(n) \ X T = X} 
denote the set of hermitian matrices, and let H (n) = {X e H(n) \ tr(X) = 0}. Note that 
su(n) = iH (n) and u(n) = ±H{n). 

Let O = and I = 1 j . We identify C with the subalgebra of R(2) spanned by 

1,1; this subalgebra has a vector space complement CO which is spanned by O and 10. Every 
matrix X G R(n) © R(2) = R(2n) decomposes uniquely as 

X = X l © 1 + X 2 © I + (X 3 © 1 + X 4 © 1)0. 

We call X 1 © 1 + X 2 © I the complex part of X and X 3 © O + X 4 © 10 the anti- complex part of 

X. There is a natural injection C(n) c >R(2n) which is given by A + Bi 1 — ► A © 1 + B © I, 

for A,Be R(n), and A + Si G H(n) if and only if A G and 5 G so(n). More generally, 
we have 

(Xi © 1 + X 2 © I + X 3 © + X 4 © I0) T = X? © 1 - xl © I + xl © + X 4 T © 10. 

Using this identity, it is not hard to show that So(2n) = H (n) ®S(n) ©CO. The corresponding 
decompositions into SU(n)-modules is 

S (2n) = su(n) © S 2 V. 
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where V = C™ is the natural module (the symmetric power of V being taken over C!). Clearly, 
su(n) is an irreducible real SU(n)-module for all n > 2. By Onishchik-Vinberg [33] p. 300, the 
complex module S 2 R(iri) = R(2iti) is irreducible for all n > 2; and 2i\\ is of complex type, 
e-R(27Ti) = i?(27Ti), provided that n > 3. Thus S^V is a real irreducible SU(n)-module for 
n > 3. 

6.11 Proposition Let H < SL 2n R be a closed semisimple noncompact subgroup with SU(n) < 
H, for n > 3. Then H is one of the groups SL„C, Sp 2n IR, or SL 2n M. 

Proof. We assume that f) < s[ 2n IR is embedded as in Theorem 6.5. Thus f) fl S (2n) is a 
nonzero SU(n)-module. We showed above that S (2n) = H (n)®S(n)® CO is a decomposition 
into real irreducible SU(n)-modules. The real dimensions of these two modules are n 2 — 1 and 
n 2 + n, respectively, so they are not isomorphic. Moreover, the following subalgebras exist: 

sl n C = su(n) ®H (n) and sp 2n R = u(n) © S(n) ® C6. 

Since 

[H (n), H (n)} = su(n), [S(n) ® C0, S(n) ® C0] = u(n) and 

[S , o(2n), 1 S (2n)] = so(2n) 

by Lemma 6.6, the algebras s(„C, 5p 2 „IR ) and s[ 2n M are the only possibilities for f). □ 

Now we consider the case where sp(n) < h, for n > 2. Let V = C 2n = H" denote the natural 
Sp(n)-module. The complex SU(2n)-module corresponding to su(2n) is 

su(n) ® C sl 2n C = {X G C(2n) | tr(X) = 0}. 

Thus we have to consider the complex Sp(n)-module V®^. By Onishchik-Vinberg [33] p. 302, 
V <g> V = i2(7ri) ® R(m) = R(ir 2 ) © i?(27ri) © C. The Galois group of C/R acts trivially on the 
fundamental weights, and 7rfc is of real type if and only if k is even, see Tits [43] p. 34. Thus 7r 2 
and 2ttx are of real type. Moreover, R(2n 1 ) = sp 2n C, and we obtain a decomposition into real 
irreducible Sp(n)-modules 

su(2n) =5p(n) © Mj R(tt 2 ). 

The dimension of r-R(vt 2 ) is (2n + l)(n — 1), and dim(sp(n)) = (2n + l)n; in particular, the 
modules are not isomorphic. The complex sp(n)-module S(2n) © C = S 2 R(n 1 ) decomposes as 

S 2 R(tt 1 ) = R(2tt 1 ), 

see Onishchik-Vinberg [33] p. 302. Thus the decomposition into real irreducible modules is 

S 2 V = sp(n) ©sp(n) 
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and 



S (4n) ^ sp{n) © spin) © sp{n) © r-R(7t 2 ). 



6.12 Proposition Let H < SL 4n M be a closed noncompact semisimple subgroup with Sp(n) < 
H, for n>2. Then H is one of the groups SL n H, SL n H-Sp(l), SL 2n C, Sp 2n C, SL 4n R, or Sp 4 „M. 

Proof. We assume that t) < s[ 4n IR is embedded as in Theorem 6.5. Thus f) n S (4n) is a 
nonzero Sp(n)-module. Note that the subalgebras s[ n H, sp 2n C, sl 2n C and 5p 4ri R exist in sl 4n R. 
We have 



Let a G Sp(l) = Censo(4n)Sp(n) be a pure element, a + a = 0. Then a defines a real symplectic 
structure u; a on M 4n = EP by u a (u, v) = Re(u T av). Thus we see that all copies of sp(n) < So(4n) 
are conjugate under the group Sp(l). In view of Lemma 6.6, we immediately obtain the following 
results. 

If h n S (4n) = r j R(2tt 1 ), then sl n M < f) and thus f) = sl n U or h = sl n M © sp(l). 
If h n S'o(4n) = sp(n), then () in conjugate to the algebra sp 2n C. 
If f) n So(4n) = sp(n) © R i?(27ri), then f) is conjugate to 5( 2n C. 
If f) Pi So(4n) = sp(n) ©sp(n), then f) is conjugate to f) = sp 4n R. 
If f) n SoiAn) = S (4n), then fj = sl 4n M. 

Thus, if t) fl 6*0(471) contains sp(n) © r-R(vt 2 ) or sp(n) ©sp(n), then su(2n) < f), and we 
showed in Proposition 6.11 that in this case either f) = sp 4n M or h = sl 2n C. 

There are no other cases to be considered. □ 

Finally, we discuss the low dimensional cases. 

6.13 Proposition Let H < SL 4 M be a closed semisimple noncompact Lie group containing 
SU(2). Then H is one of the groups SL 2 C, Sp 4 M, or SL 4 M. 

Proof. As we noted before Proposition 6.11, we have a decomposition S (A) = su(2)Q)S 2 V, 
where V = C 2 = R(ni) is the natural module (the symmetric power is taken over C). Moreover, 
S 2 R(7n) 2* R(2m), and /3 27ri = R. Thus Rj R(2vri) = su(2), and 5 (4) = su(2) © su(2) © su(2). 
Similarly as in Proposition 6.12, the centralizer Sp(l) = Cen SO ( 4 )SU(2) acts transitively on the 



s[ n Hn5 (4n) 
sp 2n C n 5 (4n) 

sl 2n CnS (4n) 
sp 4n RnS (4n) 
sl 4n M n S (AN) 



sp(n) 



sp(n) © Rj R(7t 2 ) 
sp(n) ffisp(n) 
5o(4n) 
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copies of su(2) in 5*0(4). Thus we have only the cases 

t) f) S (4) = su(2) and h = sl 2 C 
fj n 50(4) ^ su(2) © su(2) and f) ^ sp 4 R 
f) n 50(4) = 5 (4) and fj = sl 4 R. 

□ 

We summarize these results as follows. 

6.14 Theorem Let G < GL m R he a closed subgroup, for m > 3. Suppose that G acts 
transitively on the projective space RP m_1 . Let L = [G, G] denote the commutator group of 
G. Then G < Nor GLmK (L) , and there is a split short exact sequence 

1 L Nor GLmR (L) N 1. 

The following list gives all possibilities for L and the factor group N = Nor GLmK (L) / L. We put 
R> = {r e R I r > 0}. 



L 


m 


N 


SO(2n) 


2n 


R> 


SO(2n + 1) 


2n+l 


R* 


SU(ra) 


2n (n>3) 


c* 


Sp(ra) 


4n 


SO(3) -R> 


Sp(n) • Sp(l) 


4n 


R> 


G 2 


7 


R* 


Spin(7) 


8 


R> 


Spin(9) 


16 


R> 


SL 2n R 


2n 


R> 


SL 2n+1 R 


2n + l 


R* 




2n 


R> 


SL„C 


2n 


C* x Z/2 


Sp 2n C 


An 


C* x Z/2 


sl„h 


An 


SO(3) -R> 


SL n H • Sp(l) 


An 


R> 


Spin 9)1 R 


16 


R> 



Proof. The possibilities for the group L were determined in Propositions 6.9-6.12. In each 
case, it is not difficult to determine the normalizer and to construct a splitting of the exact 
sequence. □ 
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The proof of the next lemma is straight-forward; it can be used to derive a list of all connected 
transitive groups. 

6.15 Lemma Let S < W be a closed noncompact connected 1-dimensional subgroup. Up to 
conjugation, S is of the form 



for some real number a G R. The group is central in M* if and only if a = 0, and S = R>. 

Proof. Let s < R © su(2) denote the Lie algebra of S. After conjugation with a suitable 
element g G HI*, we may assume that s < R © so (2) = R © iR. Let (x, iy) G s be a generator. 
Then exp(tx, ity) = e tx e lty . Since we assumed that S is not compact, x ^ and we may put 



In the 2-dimensional case, there are the following possibilities. 

6.16 Lemma Let H < SL 2 R be a connected group acting transitively on the nonzero vectors. 
Then H = SL 2 R, or H is conjugate to C* < SL 2 R. 

Proof. We have 3 = dim(SL 2 R) > dim(iL) > 2, and H is reductive. Thus H is abelian if 
H ^ SL 2 R. In the abelian case, H acts regularly and is thus homeomorphic to C*; in particular, 
it contains a torus SO (2). The connected centralizer of SO (2) is C* = Cen S L 2 ]RSO(2). □ 

Combining the results of this section, we obtain the following final result. 

6.17 Theorem Let H < GL m R be closed connected subgroup which acts transitively on 
R m \ {0}. Up to conjugation, H is one of the groups listed in (a), (b), (c) below. 

(a) [H, H] is compact and m > 3. 



S = S a = {e t{1+ia) | t G R} = {xe ialn{x) 



x G R>} 



(x,y) = (l,a). 



□ 



[H,H] 



m H 



SO(n) 
SU(n) 
Sp(n) 




2n SU(n)-S a , SU(n) • C* (+) 
An Sp(n)-S a , Sp(n) • C* (+) 



Sp(n) • Sp(l) 
G 2 



4n Sp(n) • H* 
7 G 2 • R> 



Spin(7) 
Spin(9) 



8 Spin(7) • R 
16 Spin(9) • R 



Here, a can be any real number. 

(b) [H, H] is noncompact and m > 3. 



30 



[H, H] | m H 



SL n R n SL n R, SL n R • R> 

SL„C 2n SL n C, SL„C-U(1), SL n C • S a , GL n C (+) 

SL n H 4n SL n H, SL n H-U(l), SL„H • S a , SL„H • C* (+) 

SL n H-Sp(l) 4n SL n H-Sp(l), SL„H • H* 

Sp 2 „R 2n Sp 2n R, Sp 2n R-R> 

Sp 2n C 4n Sp 2n C, Sp 2n C-U(l), Sp 2ri C • S a , Sp 2n C • C* (+) 

Spin 9il M 16 Spin 9jl M, Spin 9jl R • R> 



Again, a can be any real number. 



(c) For m — 1, 2 there are only the following possibilites. 



m 



H 



1 

2 



R* 
C* 

SL 2 R, SL 2 R • R 



> 



(+) 



(d) If H preserves a complex structure on R m , then H is one of the groups in (a), (b), (c) 
marked with (+). 

Proof. For m > 3, the possibilities for the semisimple group L = [H, H] are determined in 
Theorem 6.14. A direct inspection (combined with Lemma 6.15) yields the list. For m = 2 we 
use Lemma 6.16, and the case m — 1 is trivial. Finally, (d) follows by direct inspection of the 
actions. □ 

Using the results in this section, it is not difficult to obtain a list of linear groups acting 
transitively on the point set FP fe of a projective space over F = C, H, see Volklein [47] Satz 2. 
Also, the possibilites for closed, but not necessarily connected groups can be determined using 
6.14. We leave this to the reader. 

7 Locally compact Moufang sets 

A Moufang set is a tripel (G, U,X), where G is a 2-transitive permutation group acting on X, 
and U < G x is a normal subgroup of a stabilizer G x acting regularly on X \ {x}. Moufang sets 
were introduced by Tits in [46]; they are also known as split doubly transitive groups. Note 
that the special case G x — U is the same as a sharply 2-transitive group. See also Kramer [29] 
Section 1.8. 

In this section, we determine all Moufang sets, where G is an effective 2-transitive Lie group 
and U is closed in G x . We call such a Moufang set a locally compact and connected Moufang 
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set. According to the classification of 2-transitive Lie groups, we distinguish three cases: the 
case where G° is simple and of rank 1, the case where G° is simple and of higher rank, and the 
affine case where X = W 1 . 

Recall from Section 3 the Iwasawa decomposition of a simple Lie group 

H — KAU 

and the corresponding minimal parabolic 

B = K AU, 

where K Q = Cen^A) is the reductive anisotropic kernel. 

7.1 Proposition Let G be a 2-transitive Lie group, with G° = H simple and of rank 1, as in 
Theorem 3.3 (a). As above, let U denote the unipotent radical of a minimal parabolic B C H. 
Then (G, U, H/B) is a locally compact connected Moufang set. 

Proof. This is clear from the Iwasawa decomposition. Let X = G/G x = H/B, and let 
y G X \ {x}. Then H x>y = B y = K A. The group U is normal in B = H x , and intersects the 
two-point stabilizer K A trivially. Thus it acts regularly on X \ {x}. □ 

If G° is a simple 2-transitive Lie group of rank at least 2, then there is no way of making X = 
G/G x into a Moufang set. The following proof was pointed out by Hendrik Van Maldeghem, 
replacing a more topological (and more complicated) argument of mine. Let X be the point set 
of a desarguesian projective space of rank at least 2, or the point space of a projective Moufang 
plane. Let G be a group of automorphisms of the projective space, containing all elations. Then 
(G,X) cannot be made into a Moufang set. To see this, let p G X and assume that U < G p is 
a normal subgroup acting regularly on X \ {p}. Let u EU , and let r be an elation with center 
p. Then uru^ 1 is also an elation with center p, and so is the commutator [u,r]. If we choose 
u, t in such a way that u does not fix the axis of r (which is possible, since the rank of the 
projective space is at least 2), then [u,r] G U is a nontrivial elation with center p. Since U is 
normal, U contains all elations with center p. These elations form an abelian normal subgroup 
of G p which is, however, not regular on X. 

7.2 Proposition None of the groups in Theorem 3.3 (b) can be made into a Moufang set. □ 

Finally, we consider the question of uniqueness in the case where H = G° is simple and of rank 
1. The question is thus if G x admits a regular normal subgroup V different from U. Since 
X \ {x} is connected, we have V < B = H x = K AU. So V n K A = 1 (by regularity of V) and 
K Q AV = B (by transitivity). Let u denote the Lie algebra of U , and the Lie algebra of V. We 
have to prove that = u. We decompose the Lie algebra b of B into irreducible -?T A- modules. 
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As a A" A-module, t © a decomposes as [t , to] © Cen(6 ) © Direct inspection shows that no 
i^o^-sukmodule of u is isomorphic to a A" A-submodule of 60 © <*■ This proves uniqueness of u. 

7.3 Lemma Each of the groups in Theorem 3.3 (a) is in a unique way a locally compact and 
connected Moufang set, i.e. the data (G,X) determine the closed regular normal subgroup 
U <G X uniquely. □ 

It remains to consider the case where X is noncompact. Then G is a semidirect product 
G = G x ix IR m , and U < G x is a normal subgroup acting regularly on the nonzero vectors in M. m . 
It follows that the chosen maximal compact subgroup K C U acts regularly on the sphere S m_1 . 
Direct inspection of the list in Theorem 6.1 shows that this happens only for m = 1,2,4, and 
\U, U] is one of the groups 1, SO (2), Sp(3). We obtain the following result, which is originally 
due to Kalscheuer [25] and was re-proved by Tits [40] and Grundhofer [16]. 

7.4 Theorem Let U < GL m M be a closed subgroup which acts regularly (i.e. sharply transi- 
tively) on R m \ {0}. Then m = 1, 2, 4, and U is one of the groups W, C* , M* = Sp(l) • S , or 
Sp(l) • S a , for a^O. 

Proof. The result follows by direct inspection of the tables in Theorem 6.17. □ 

Since U < H is a normal subgroup, it remains to determine the normalizer NorQL m R(f^)- The 
following table shows the corresponding quotients Nor GLm iR(t/)/£/. 



u 


m 


Nor G L m i 


(U)/U 


w 


1 


1 




c* 


2 


Z/2 




e* 


4 


SO(3) -I 




Sp(l) • s a 


4 


M> (a 


^0) 



Using this, it is not difficult to determine the locally compact and connected Moufang sets for 
noncompact X. Also, it is easy to see that the pair (G,X) determines U for m — 1,2. This 
is not true for m = 4: if H = Sp(l) • C* is the group consisting of all maps x 1 — ► hxc, for 
h G Sp(l) and c G C*, then Sp(l) • S a — U < H is a regular normal subgroup for any choice of 
a G K. 
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